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ABSTRACT 


10700 CTS 01 015 paper will be centered around 
the issue of whether the Navy should restructure the Opera- 
tional Test and Evaluation Force (OPTEVFOR) along the lines 
of the Army's and Air Force's independent test agencies. The 
paper begins with an overview of the weapon system acquisi- 
tion process. It tnen proceeds to demonstrate how test and 
> luacriuobaCPEEANTIKS"invo the acquisition process. Finally: 
a description and evaluation of the each of the service's 
independent test agencies are presented. 

The researcher suggests that the perceived problems are 
not caused by OPTEVFOR's present structural arrangement, but 
771 the result of OPTEVFOR's philosophy. The final conclu- 
7700 is that OPTEVFOR should not change its existing struc- 
mice, instead it needs to expand upon its current emphasis 


on increased communication. 
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T; INTRODUCTION 

Weapon system acquisition, the development and procure- 
ment of military systems, is a matter of widespread and 
increasing concern. Government Reports, Congressional 
hearings, as well as articles in the press, have suggested 
that the acquisition process is not working as well as it 
should. A few examples of these shortcomings are: that 
development fails to produce a satisfactory working design, 
that major systems incur large cost overruns, and that parts 
are over priced. 

Testing of new weapon systems is one of the Department 
of Defense's (DOD) key controls in the process of acquiring 
these complex and expensive systems. Adequate test and 
evaluation (T&E) of these weapon systems is of paramount 
importance to ensure maximum return for the dollars spent 
to procure the systems. 

Significant improvements have been accomplished in the 
test and evaluation portion of the acquisition process. In 
earlier years, test and evaluation was often accomplished 
when or where time allowed and usually by personnel who were 
not trained in the techniques of testing. After the July 
1970 Blue Ribbon Defense Panel Reports, test and evaluation 
policies were significantly altered. Separate independent 
test agencies were established within each service. These 


agencies, although organized differently, had one thing in 


Cenone maintained its testing independent fron the 
see Oper, reporting directly to its own service 
headquarters. These independent agencies perform what is 
commonly called Operational Test and Evaluation (OT&E). They 
check to see if and how well the system will function in an 
operational environment. 

dow Operational Test and Evaluation is being implemented 
has been one of the central issues identified in numerous 
committees, conmissions and reports on weaknesses in weapon 
System acquisition. The issues addressed in these studies 
nave been consistently repeated. A report done in February 
1977 by the Defense Science Board identified one 
25 eante Ssnortcoming=in OT&E. The report stated that it 
mer Very important for all participants in the acquisition 
BHB “OO participate throughout the entire acquisition 
mmeoeess. AL the time of this report this interaction between 
the developer and service OT&E organizations did not exist. 
o. 1: p.7] 

Directives and instructions have been issued which 
should have corrected some of the major OT&E problems. These 
instructions however, have not been effectively implemented 
in the Navy, and this has resulted in the Navy's OT&E 
practices remaining essentially unchanged with little 
interaction between the developer and the independent 


operational test agency. This course of action has been one 


of the reasons for an increase in tne Denn 
acquisition process,» which in turn has “ineorease 073 
overall costs of the process as welll 

As a result of the flavy"s reluctance for interacerTeoni:. 
question has been asked about the present structure of the 
lavy's independent operational test agency. [his question 18 
should the Navy restructure its Operational Test Wana 
Evaluation Force (OPTEVFOR) along the lines of the Arn NE 
tne Air Force? Presently the Army and Air Force independent 
operational test agencies act as a manager, aliowing the 
user to do the actual operational testing. İn€” 15002 
Operational Test and Evaluation Force on the other hand) 
acts as a surrogate for the fleet (user) and does alla 
Operational Test and Evaluation. It is felt by some, 722 
by restructuring. the user's inputs could be introduced mnia 
the process on an earlier and continuous basi s mm 
contention is that the restructuring vould also enormour”. 
reduce the size and cost of OPTEVFOR and additionallys"nouae 
provide a trained first fleet unit at the end O ema. 
Operational Evaluation (OPEVAL). An additional poten" 
benefit espoused would be an earlier Initial Operation 
Capabi1ilitv (IOC). [Re иә) 

The main thrust of this thesis will center around 
issue of whether the Navy should restructure OPTEVFOR along 


the lines of the Army and Air Force. However, be or msn 


issue can be properly addressed, a foundation nas to be 
илки ити елә wile consist of an overview of the 
weapon system acquisition process, how T&E fits into tne 
sitio” process. and finally a description and 
evaluation of eaeh of the service's independent test 


agencies. 


11. DEFENSE ACQUISITION PROCES. 


The issue of whether to restructure the Navy's Opera- 
tional Test and Evaluation Force can not be properly 
addressed without a basic understanding of the DoD's weapon 
system acquisition process and how the Service's T&E program 
fits into the overall acquisition scheme. 

The present method of managing the weapons systems ac- 
quisition process in the DoD, emerged as a result of a study 
by the President's Blue Ribbon Defense Panel (BRDP) done in 
1970. As a result of recommendations of the BRDP, the 
Department of Defense Directive 5000.1 was promulgated and 
released in 1971. The next item to be released in this 
evolution was the issuance of Office of Management and 
Budget (OMB) Circular A-109, which was released in 1976. The 
policies and guidelines in these documents form the basis 
for all the subsequent directives and instructions regulat- 
ing the acquisition of weapon systems. These directives and 
instructions enable the Secretary of Defense (SECDEF), who 
is aided by the Joint Requirements and Management Board 
(JRMB), formerly called the Defense System Acquisition 
Review Council (DSARC), to guide and control the development 
and production of major weapon systems through a series of 
acquisition phases, milestone reviews, and decision points. 


The process is illustrated in Figure 1 İRef: 2007007 
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It is the Secretary of Defense who makes the deCiSIGONWNI 
to whica new systems will De designated a major system. Tne 
decision to designate any system as major is based upon: 


a. Development risk, urgency of need. or other tems 
interest to the SECDEF. 


Do Joint acquisition of a system by the DOD and Pe 
sentatives of another nation, or by two or more DOD 
Components. 

cz Ihe estimated requirement for the system's research, 
development, T&E, procurement (production) and opera- 
tion and support resources. A Justification for MSN 


System New Start (JHSNS) is required for allmacon 
tions for which the DOD Component estimates COSINA 
exceed $200 million (FY80 dollars) in RDT&E funds or 
$1 Billion (FY80 dollars) in procurement (productions 
Гаоназәамәг both. 

d. Significant congressional interest: ¡(ON 

It should be noted that all veapon systems acquisitions 
go through a similar process to the one outlined heremum 
major systems. The only difference is that decisions are 
made at lower levels within the Services. 

There are four distinct phases and milestones 1292505 
system acquisition Process: The actual acquisition o nE 
new system begins with either a decision to establish new 
capabilities in response to a technologically feasible op- 
portunity, an identified deficiency in amwmex SANA 
capability, a significant opportunity to reduce the Dobbie p 
of ownership, or in response to a change in the national 


defense policy [Ref 4: pp.3-4]. This part of the processes 


called Milestone 0. 


By preparing a Justification"for Major System New Start 
"JMSNS), the Services document the need for the new system 
and submit it into tne Planning, Programming, and Budget 
| ZSEENWACPPBS) processwalong with the Service's” Program 
Objective Memorandum (POM) for tne year in which the funds 
are requested. The Secretary of Defense may than Sanction 
the JMSNS in the Program Decision Memorandum (PDM). dins 
action, designated Milestone Ü, authorizes the Service to 
initiate the program when funds are available and moves the 
program into the next acquisition phase which is called 
moncept Exploration (CE). 

sq che Concept Explorationwphase, several contracts are 
awarded to industry to identify and investigate alternative 
System design concepts that will satisfy the mission need. 
At the conclusion of this phase, the Program Manager (PH) 
makes his/her recommendation for those concepts which have 
tne potential for further development and evaluation and 
should be carried forward into the next phase of the ac- 
aDisition process which is called the Demonstration and 
Validation (D&V) phase. This recommendation is documented in 
pae System Concept Paper (SCP). 

Tne SCP súübmitted to the Joint Requirements and 
Management Board (JRMB) and then forwarded, if approved, to 
DOEDSECDEP. The SECDEF uses the SCP to make the Milestone I 


decision . 1157 000601 7) Cto proceed is provrded in tne 


iLə 


Secretary of Defense Decision Memorandum (SDDM). Approval 
signifies a validation of the requirement and is authoriza- 
tion to proceed with the DöV phase with the most promising 
concepts [Ref H: p. 4]. 

An additional document developed during the CE phase, 
is the Test and Evaluation Master Plan (TEMP). The) Sit) eee 
short, concise master plan for T&E. Its purpose is to ICIS 
tify all required T&E resourcesü facilitate long-range plan- 
ning,» programming, and budgeting including that of adequate 
numbers of test hardware items and specialized major range 
and test facilities¢s ensure accomplishment of adequate T$E8 
and eliminiate redundant testing. The TEMP forms the basic 
contract between the Development Agency (DA) which is 
responsible for the Development Test and Evaluation (DT&E) 
and the independent test agency,» who is responsible for the 
Operational Test and Evaluation (OT&E). While the" INCA 
version of the TEMP, which is required at Milestone I, will 
lack many specifics, through a continuous revision proce 
the TEMP will develop the necessary detail. (Ref. 3:p. 3-19] 

A positive Milestone I decision allows the systeme 
enter the Demonstration and Validation phase. The purpose of 
this phase is to further develop and validate the alterna= 
tive concepts to determine which concept(s) should progress 
into the Full Scale Development (FSD) phase [Ref.3: p.1-15]. 


This phase involves demonstration of the technology to be 
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1157 25000 - SYSLem Opmcritical subsystems to verify perfor- 
ancora ate potential suitability of the concept to fill 
Bc 0 0 00 uced 06 CDİ oont, scientific and technologi- 
"EJ development is regGuired to bring the concept to its 
FUI CLON. 

It is during the D&V phase that both DT&E and OT&E begin 
Dun tne test results being used to support the Milestone 11 
decision. cu исә ии issues and areas of risk 
that have been addressed by test and evaluation, the Deci- 
52007 Coordinating Paper (DCP)/7integrated Program Summary 
5 is prepared by the Program Manager. The DCP/IPS 
provides the test objectives and measures of effectiveness 
Al acted to the satisfaction of mission need and resource 
mequarements that apply to the test activity. The PM sudmits 
20 document through the JRMB to the SE рир pí Os the 
EM one TE decision, SECDEF approval authorizes the 
Amos ram to proceed into the Full Scale Development phase 
1550). 

ik” LDƏ c some tide FD phase is to produce a fully 
tested, documented, and production-engineered design of the 
selected concept [Ref. 3: p.1-151. This phase is divided 
iso three subphases: engineering» prototype, and pilot- 
m0 .on4transition to produetion. It is during this part 
Ehe acquisition cycle that two major evaluations take 


piace. The first is the technical evaluation (TECHEVAL), the 
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purpose of which is to identify technical deficienciesann 
determine whether the design meets technical specificat Tonk 
and requirements (Ref. 5: p.2]. This evaluation is “87 
important facet of development testing, for if the system 
passes the TECHEVAL it goes on to Сһе пехС пајог еҹабһивинәи 
which is called Operational Evaluation (OPEVAL). Given the 
data from the TECHEVAL, OPEVAL proceeds to test the system 
for demonstration of operational effectiveness and operas 
tional suitability, in addition to verification of PONE n 
problems discovered in the TECHEVAL [Ref. 5: p. 4]. These 
formal evaluations are performed in order to certify readi- 
ness for the production phase. 

Tne final milestone decision point, Milestone III, is 
to decide whether or not the system is to enter into produc- 
tion and deployment. The PM documents the results of the 
roD phase, including testing, and plans for future Cesta 
are assessed again. The PM then sends his recommendations in 
an updated DCP/IPS through the JRMB to the SECDEF 1005350 
decision. Normally, however, the Secretary of Defense 
delegates the Milestone III decision to the Secretary of the 
branch of the Service that is responsible for the system. 
For less than major systems, this decision in some cases is 
delegated to the lowest level in the organization at which 
overall view of the program rests. This is the point wep 


test results play their most critical role because ə 
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51:57 PRE aecIilS1ON maker in his decision whether to approve 
ee deny production. 

An affirmative Milestone III decision allows the 
system to enter the Production and Depioyment Phase. One or 
Lore Contractors ane awarded a production contract for 
either low-rate or full rate production. Even after the 
system is being produced, testing does not stop. The system 
WS assessed in new environments, in different platform 
applications, in new tactical applications, or against new 
threats. 

Besides the aforementioned fixed milestones, additional 
program reviews can be held at any point in the program wnen 
major problems arise. E of the information at the reviews 
comes from the test and evaluation results and is used to 
assess the programs progress and its likely SUecęss 
lef 6:p. 88]. Depending on the findings, a program may 
advance to the next phase, be canceled or just held at its 
present pnase. 

in summation, the acquisition process that is presented 
here is a very simplistic overview. This process, though 
conceptually simple, in fact is extremely detailed in its 
requirements. Test and evaluation is an essential player in 
the Department of Defense's acquisition process, in that 
the T&E results help to determine the future outcome of a 


program. 
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III. WHAT IS TEST AND EVALUATION 2 

Test and evaluation is a fundamental and integral part 
of the acquisition process. Although the terms "test" and 
"evaluation' are used jointly, they do represent two dis- 
tinct and unique functions. One definition of TEST is " a 
critical examination, observation or evaluation; a trial or 
eroup of trials" [Ref. 7:p. 2]. Evaluation, on the ones 
hand concerns the review and analysis of data produced 
during testing or use of the item. 

The Department of Defense is generally concerned with 
three types of test and evaluation: Development Test and 
Evaluation (DT&E), Operational Test and Evaluation (OT&E), 
and Production Acceptance Test and Evaluation (PAT&E) 
[Ref.8: p.2]. Figure 2 is a summary overview of the acquisi- 
tion process, and it illustrates where these three types 


of test and evaluation fit into the acquisition process 


[Ref 39 p. ДИ. 


A. DEVELOPMENT TEST AND EVALUATION (DT&E) 

DT&E is that test and evaluation conducted to 
demonstrate that the engineering design and development 
process is complete, the design risks have been minimized, 
the system will meet the specifications, and to estimate the 
system's military utility when introduced [Ref. 9: p. 7700 


Development Test and Evaluation is planned, conducted and 
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Acquisition Process. 


monitored, by the Developing Agency (DA) Of them ee 
Departments or other Defense Agencies, with the resul 
being reported to the responsible Military Service M 

DT&E is required for all acqulsit101792207 5” Lt Gam 
be oroken down into three major phases: DT-I, DT-II, and 
DT-III. The specific objectives of each phase are developed 
by the DA and are published in the Test and Evaluate 
Master Plan 1150777 

DT-I is conducted during the Demonstration and Valida- 
tion Phase to support the Milestone II decision. TCS TOTAN 
cipal purpose is to demonstrate that all technical m 
have been identified and that solutions to the risks are in 
hand (Rer, 9:p. 7-8]. 

Conducted during the Full Scale Development Phase, DT-II 
is used to support the production and deployment (Milestone 
111) decision. DT-II demonstrates that the design meets its 
Specifications in performance, compatibility, Suppo. 
tability, survivability, safety, and human factors [Ren aa 
p. 7-9]. 

The Technical Evaluation (TECHEVAL) is the final phase 
of DT-II. It is conducted to determine whether o Seni Eo 
system is functioning in a technically acceptable manne 
whether it meets technical and design performance specifica- 
tions, and to determine if it is ready for the Operational 


Evaluation (OPEVAL). The Developing Agency (DA) has the 
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MAT mes posibili for planning the test program and ob= 
Ii thelFkoresuPcse ATEeP Lhe TECHEVAL; the DA”certifies 
ias system 1s ready ror the Operational Evaluation 
17:77:00)” However, OPEVAL may not commence until tne 
Secretary and the independent test agency of tne concerned 
BE Den of the senvice accepts the DA's certification of 
readiness for OPEVAL. [Ref. 9: p. 7-9] 

una  nalepudsemor DISEOMNSEDDU-TITI. It ís conducted after 
B= production and deployment decision (Milestone 111). DT-- 
1511 is used to verify the correction of design deficiencies 
Bund during TECHEVAL or OPEVAL, and that any improvements 


that were made to the system are effective. 


57 OrE RATIONAL TEST AND EVALUATION COTZE) 

Operational Test and Evaluation is the test and evalua- 
tion conducted by an independent agency to determine the 
prospective system's military utility, operational effec- 
mmveness (including compatibility. interoperability, 
meriability, and maintainability); and logistic and training 
mequirements (Ref. 9: p. 7-9]. Since OT&E is conducted under 
e aro realistic operating conditions as possible, 
needed modifications to the system do become apparent during 
ms process. The operating personnel, as well as required 
Support facilities, are typical of those expected to operate 


and maintain the system when it is deployed. 
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OT&E can be broken down into two categories: 10557 
Operational Test and Evaluation (IOT&E),; which uds ia 
conducted prior to the production and deployment deca cmm 
and Follow=on Operational Test and Evaluation (FOTE) V ee 
is all OT&E which is conducted after the prodüuction W 
deployment decision. OT&E can also be divided into four 
phases (two in IOUSE and two in b om EUM 

OT-I is the first phase of Initial Operational ieee 
and Evaluation (IOT&E), and is conducted durinEWIE 
Demonstration and Validation Phase to support the full-scale 
development decision (Milestone II). Its objectives ara 
provide an early assessment as to whether or not, based on 
the system's effectiveness, the project/program should “2255 
tinue on its development (Ref. 9: p. 7-9]. 

The other phase of IOT&E is called OT-II, and is con- 
ducted during the Full-Scale Development Phase. The OPEVAL 
is the culmination of OT-II. The objectives of OT-11 arENUNB 
demonstration of the achievement of program objectives for 
operational effectiveness and of operational suitability. 

The first phase of  Follow=on Operational des tala 
Evaluation (FOT6£E) is called OT-III, and is conducted atcem 
the production and deployment decision. This testing sms 
before production systems are available. Normally 1:555 


done using the same preproduction system that vas used in 


22 


ERCAOPENMAE. > Wie specific objectives of OT-III include: test- 
0 1165 OT corTrections that were incorporated, comple- 
tion of any deferred or incomplete lnitial Operational Test 
27 EvaluaLion (IOTCE), amewevaluatiońn of any new tactics 
development. 

OT-IV is the second phase of FOT&E, and it is conducted 
sm the production system. One of the initial objectives of 
OT-IV is the demonstration of the achievement of program ob- 
jectives for production system operational effectiveness and 
ee atonal suitability, especially reliability, main- 
amnabiljity and logistic supportability. Additional objec- 
tives are OT&E of the system in new environments, in new 


applications, or against new threats. 


EGRODUCTION ACCEPTANCE TEST AND EVALUATION (PAT&E) 
Production Acceptance Test and Evaluation is conducted 
upon production items to ensure that the items procured meet 
WI Nsoecifications of the procurement contract. Successful 
"oopletion constitutes the recognition that the item 
fulfills the requirement for which it was produced. PAT&E is 
the responsibility of the Developing Agency and begins in 
Ro transition to production section of the Full-Scale 
Development Phase and continues through the Production and 
Deployment Phase [Ref. 9: p. 7-9]. 
The single most important message that comes out of tne 


Mest and evaluation evolution is that any new acquisition 


7 


program must be capable of meeting or exceeding moe am 
world goals expected of it in the actual or Com 
Simulated operational environment prior to any eee 
production commitment. The information obtained by P 
used to give a better understanding of the sys Cemmis 
capabilities, to identify shortcomings, Co qes 
improvements, and to assist in designing the Syst 
future replacements. Additionally, an important facet of the 
program is to verify tne ability to support and maintammm 
System and to develop requirements for training personnel in 


the operation of Che System: 
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IV. THE SERVICES' INDEPENDENT TEST AGENCIES 
FOR OPERATIONAL TEST AND EVALUATION 

Having established a base from which to work, each in- 
dividual service will be reviewed to see how they are 
structured to handle the requirement for independent test 
and evaluation. However, before this is undertaken, the 
issue of what prompted the drive for the independent test 
agencies will be presented. 

During the period prior to the President's Blue Ribbon 
Defense Panel Report (released on 1 July 1970), defense sys- 
tem planners and developers received little attention from 
Congress on the methods they used to develop and test the 
new systems. At that time, most of the controversy sur- 
rounded the actual need for the weapon system and the cost 
of placing the weapon system into the defense inventory. The 
concern of whether the weapon system could in fact perform 
to the designated specifications or was actually ready for 
production was very small outside the Department of Defense. 
Most of this lack of concern can be attributed to a general 
feeling of satisfaction within the Congress that the govern- 
ment research, development, test, and evaluation (RDT&E) 
side had been very successful since the United States was 
winning the intercontinental ballistic missile and space 
race. It was during this period in time that the U.S. placed 


a man on the Moon. Another factor that kept the Congress's 


zə 


attention was the war in Vietnam. With the change of Presi= 
dents and a change in attitudes toward economic conditions 
and domestic issues, the Congress intensified Yes 5007 
of Defense expenditures, in particular the area of Mad 
weapon system acquisition. It vas tnis intensi eo mn 
the Congress that brought about the dramatic change 1177022 
military services! test and evaluation po Mibi 

The President's Blue Ribbon Defense Panel (BRDP) was one 
of the major driving forces behind the one hundred eci 
eighty degree change in attitudes. In the BRDP report the 
Department of Defense's operational test and evaluation 
practices were specifically addressed as a significant 


problem area. The following excerpt points out the concedi 
of the panel: 


There has been an increasing desire, particularly 

at tne OSD level, to use data from OT&E to assise 

in the decision-making process. Unquestionaul yaa 
would be extremely useful to replace or support 
critical assumptions and educated EUĘSSĘSTUUEN 
quantitative data obtained from realistic and relevant 
Operational testing.: 


Unfortunately, it has been almost impossible to 
obtain results which are directly applicable to tne 
decision or useful for analysis. Often, test data do 
not exist. When they do, they frequently are derived 
from tests which were poorly designed or conducted 
under insufficiently controlled conditions Compania 
valid comparisons. It is especially dif ficulcumgs 
obtain test data in time to assist in decision-making. 
significant changes are essential if OT&E is to real- 
ize its potential for contributing to importan Ki Gw 
Sions, particularly where the tests and the decisions 
must cross Service lines. [Ref P lO DON 


Secretary Packard, then tne Deputy Secretary of Defense, 
was tne driving force within the Departnent of Defense for 
anne the test and evaluations policies. He personally 
15207 series Of memoranda te the Services in which he 
pequested that the services arrange for OT&E to be managed 
and conducted by an independent agency separate from the 
developing command and which reported their results directly 
bo che Chief of the Service. 

mie iordal directive came in the form of two Department 
77 Defense Directives (DODDs?. The first was Department of 
55 onsc Directive 5000.1, which officially tied OTEE with 
me Joint Requirements and Management Board (JRMB), then 
sued DSARC, Milestone III decision point. 

Test and evaluation shall commence as early as pos- 
55016. A determination of operational suitability: 
including logistic support requirements, will be made 
Bor to large scale production commitments, making use 
of the most realistic environment possible and the best 
representation of the future operational system avail- 
able. The results of this operational testing will be 


evaluated and presented to the DSARC at the time of tne 
production decision. [Ref. 11: para.III, C.1] 


The second directive was DODD 5000.3, which provided ad- 
EN onal and more specific test and evaluation policy 
ebidance than that of DODD 5000.1. 


In each DOD component there will be one major 
agency separate and distinct from tne developing/ 
procuring command and from the using command which will 
Be responsible for OT&E and which will: 
777 Dort the results of iLs independent test 
and evaluation directly to the Military 
Service Chief or Defense Agency Director. 
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b. Recommend directly to 1tf57"1110477777777 
Chief or Defense Agency Director the ac- 
complishment of adequate OT&E. 

c. Insure that the OT&E is effectively planned 
and conducted [Ref.128: para m a 

The stage nad now been set with the official Office mon 
the Secretary of Defense direction for the formation one 
independent testing agency within each service. 

The Blue Ribbon Defense Panel Report should be con- 
sidered the catalyst, which resulted in a major chant SENi 
how the Services viewed their test and evaluations = 
tivities. The most notable change came in the area of OT&E. 


An examination of how each branch of the service incor- 


porated this new philosophy follows. 


A. ARMY 

During the period 1962-1971, the testing of systems and 
equipment under development and in-process of acquisitiong 
the Army was accomplished almost totally within eneee 
Material Command (AMC). The Army Material Command assigned 
tae test responsibilities to the Commanding General, United 
States Army Test and Evaluation Command (TECODE 

As the Army"s principal material testing organization: 
the Test and Evaluation Command was assigned the basic mis- 
sion of providing the decision makers witn unbiased indepen- 


dent appraisals of Army material. The Army chartered tne 
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TECOM to reduce the time frame between design and production 
Miro eliminate duplication of eifort througń integrated 
51842 and better coordination. 

To accomplish what the charter mandated, TECOM employed 
botn engineering tests and service tests. Engineering tests 
per cmeonducted to determine if the system really met the 
material need. Service tests examined the human interface 
with the system. These tests determined the equipment's 
Emu -ability for tne Army's use and for release to produc- 
tion. Actual operational units were employed to conduct 
Bests in a tactical environment and in general possessed 
1575 cor tnhe"emaracteristics of present OT&E practices. 
However, tnese tests were often conducted after the manage- 
ment decision to produce the system in numbers and even 
Ever the deployment for Army operational use. 

Since the Test and Evaluation Command was affiliated 
with the Army's Material Command developers», a new and inde- 
pendent command was necessary in order to meet the require- 
ments of the Department of Defense Directive. As a result, 
the Operational Test and Evaluation Agency (OTEÀ) was estab- 
150-060 on 25 September 1972 at Fort Belvoir, Virginia. 
Through their charter from the Secretary of the Army. OTEA 
was given the management responsibility for major and non- 


major systems. [Ref. 13: pp. C-1 - C-3] 
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The Commanding General of OTEA reports to the Chief of 
Staff of the Army. The mission and functions of OTEA are as- 
signed by Army Regulation (AR) 10-4, "U.S. Army Operational 
Test and Evaluation Agency". The mission of OTEA "is to 
manage all user testing, operational testing (OT), force 
development testing and experimentation (FDTE), and joint 
user testing directed by the Office of the Secretary of 
Defense" [Ref. 14: p. 2]. The specific tasks which are as- 


signed to OTEA are summarized as: 


a. Plan, direct and evaluate the operational testing of 
all major and selected non-major systems. 


b. Coordinate the operational testing of other non- 
major systems. 


c. Manage major and coordinate non-major FDTE. 


d. Coordinate Army participation in the planning and 
execution of Joint OT&E. 


e. Provide a strong focal point organization to keep 
the Developing Agency (DA) and the Office of the 
Secretary of Defense (OSD) fully informed on the 
Army's OT&E needs and accomplishments. 
The original manning level of OTEA was set at 120 person- 
nel, which included: 53 officers, 2 enlisted, andam i 
civilians. In 1975, the strength was increased to 250 per- 


sonnel with 125 officers, 20 enlisted, and 105 civilians. 


Today, its strength is 256 personnel with 122 officers, 8 
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527) 060 and 1260 civilians İRef. 15). Figures 3 shows 
mu Serelationshnip Witnin tne Army's Test and Evaluation 
peta canton. Figure 4Jisetme organizational chart for OTEA. 

Wnen the Operational Test and Evaluation Agency receives 
EE cem irom AMC for PGE. it either appoints its ovn test 
director and team or tasks the Force Command (who is respon- 
27" .or the continental 4/.5. Army Divisions) along with 
the aterial Development and Readiness Command and/or the 
Training Doctrine Command to perform tne testing. This deci- 
sion is made based upon the size and importance of the 
“506ram/system to be tested.  OTEA's role when it retains a 
7 on for testing is to furnish three to five of its per- 
57026. to fill key positions on the team such as the Deputy 
EN ccror for the systems OT, the Chief Analyst, the Chief 
15 Collector, and the Chief Controller. İRADOC, repre- 
K Ling the user, provides the appropriate assistants in the 
areas of conceptual expertise. The remaining positions are 
filled by the testing command. If the program/system OT&E is 
ked to an organization other than OTEA, OTEA mainly 
evaluates the data generated by the testing command. OTEA 
WS currently directing/managing OT&E for 62 major and 250 


Messer category non-major systems [Ret. 10]. 
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Bay ATR FORCE 

In their attempt to conform to the DOD requirement 
Air Force initially tried to acnieve independence in opera 
tional testing by placing the full OT&E respons:0117 
the user Command. Even though this sas a step in tn6 “12022 
direction, the Air Force's effort fell short of 175 mm m 
was often subject to criticism. To correct this “12:07:22: 
the Air Force, in October of 1975, established its indepen- 
dent test agency calling it the Air Force Operational 1222 
and Evaluation Center (AFOTEC) at Kirkland Air Forco6 es 
dew tiexico. AFOTEC is a separate operating agency UNA» 
reports directly to the Chief of Staff of the Air Force 77 
addition to its headquarters at Kirkland, 4EOTEC 14:72 
additional permanently estabiished detachments at Edwards 
Air Force Base, California, Eglin Air Force Base, Florida; 
Nellis Air Force Base, Nevada, and Kapaun Air Station, West 
Germany. Along with these four permanent detachments there 
are additional field test teams at various designated sites. 
[Refs 127-302 

Tne center's headquarters and field teams include opera- 
tional, technical, analytical and test specialists. 107 
field test teams conduct their test av tne selected euE 
and from the results prepare the formal reports. These field 
teams are composed of personnel from the Air Force Test and 


Evaluation Center along with personnel from the Opera 
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anepsuUupport comnands who wili eventually receive dnd utilize 

Ene systems in every day operation. 

1? Gorce Ooperatkiehal Test and Evaluation Center's 
peated mission is "to manage the Air Force operational test 
MO evaluation program and assess the military utility and 
poerational effectiveness and suitability of major and 
EI ed non- major systems [Ref. 17: p.1]. The specific 
BRE ¿nico are assigned to  AFOTEC are summarized as: 

a. Design, direct, analyze, evaluate and report on 

Slat wei magor ands destened@=mon-major Air Force sys 

tems. 

2 Designate tne Deputy Test Director and provide (with 
augmentation from major commands) the OT&E team for 
combined DT&E/IOT“E programs retained oy AFOTEC. 

27 uobitor all non-major system OT&E conducted by 
she fajoR commańds ( e.E. TASZA SAC, MAC) through 
review and approval of tests plans and review of 
comments OM che test reports. 

d. Develop policy recommendations for Headquarters; 


snited States AIr 5Bereerapprovwad "and .suBsequent is 
plementation by all najor commands. 


e. Act as spokesman to the Joint Requirement and 
Management Board (JRMB) on matters pertaining to 
GELE, 


7— 1: rt Cońgressional requests for OlGE information 
on weapon Systems for whic Bbrocuremęent funds are 
being requested. (Ref. 17: p.1] 
SIRCE becoming Operational, the manning of AFOTEC has 
25: --sed from the initial authorization of 208: to the 


present strength of 677. This manning level includes 477 of- 


MESS. 90 enlisted, and 110 civilians. AFOTEC is 


additionally augmented by 2500 personnel from the previous 
mentioned operating and support commands [Ref. 138] Tne 
organization of AFOTEC is similar to that et hee eee 
OTEA. Figure 5 shows the overall relationship of 2500 
within the Air Force, and Figure Ó is the organization m 
Of APOPECE 

Air Force Operational Test and Evaluation Cente TER 
currently managing OT&E for 55 major and designed non-major 
programs. In addition, the center is presently moniLOGENEJ 
240 operational test efforts being done by other major USAF 


commands. [Ref. 19] 


C. NAVY 

Tae Navy had an easier time than its sister Services in 
coming up with a test agency that was independen ms ger 
developer, since the Navy already had one in existence. This 
command is call the Operational Test and Evaluation Bene» 
СОРТЕЛ УД. 

OPTEVFOR can trace its origin to the fina S n a 
Hor idulanBı e During the Okinawa campaign, the surface 
forces of the Navy came under a sustained concentrated 
attack by Japanese kamikaze aircraft. These suicide attacks 
took a great toll in ships and personnel, and a means to 
deal with the threat was urgently required. In July of 1945, 


the Composite Task Force, U.S. Atlantic Fleet was formed and 
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tasked with the development and evaluation of netnods to 
Ep Dp i altacks. [mls Was the origin of tne Navy's 
5 Ppondent test ageney. LRef.,. 13: pp.D-1 - 2-3) 

In the following years, changes were made to tne name of 
the command and to the assigned mission and tasks. These 
£ .nuzes”resulteC in a wider scope of responsibilities and 
7 panded capabilities in test and evaluation. Additionally. 
ENS ur'arly structured command was formed on the west coast 
mer the Pacific Fleet. 

To achieve the desired independence which was suggested 
in the siue Ribbon Defense Panel report and required in tne 
2  :-cotives, OPTEVPOR had to make a shift in its 
“10nasis. Tais change in mission orientation was 
accomplished in the period between late 1971 and middle 
EN voile the reorsanization did not amwvwóalve an 
mereciable Organizational restructuring or an increase in 
"Phpower allocation, it did require a different and more 
7 0:nsive mix of personnel. The Navy's "hands-on" 
Eoerational testing concept required operational officers 
555 .iznly skilled nonconmissioned officers that were 
2 00) ar vith the current missions, tactics, policies, and 
7 0cedures. İFef. 13: pp. 8-1 - D-3] 

Memo eratlional Test end Evaluation Foree is now under 
the direct command of the Chief of Naval Operations (CNO) 


BOG Che conduct of OT&E. The orzanizational relationsnip 
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that OPTEVFOR has witn tne DOD/NAVY can be seen"in 7120777 
The command headquarters for OPTEVFOR is located m e 
Norfolk (Virginia) Naval Base. For the Pac171707428 02 
operations, a separate staff under tne Deputy COHOPIEVEO Kis 
located at the Naval Air Station, Nortn island, San Diego, 
California. The function of the Deputy CONOPTEVEFOR 15 702 
as the representive of OPTEVFOR in matters in the as 
area and, when directed, with west coast operation wm 
commands (e.z. COMNAVATRPAC., COANAVSUPFACZE 

The mission of OPTEVFOR is assigned by tne Cil@ amc 
"to operationally test and evaluate specific weapon SyStemer 
snips, aircraft, and equipments, including procedures mama 
tactics, where required: and, when directed by CNO, ASS Eoi. 
development agencies in the accomplishment of necessi 
Developmental Test and Evaluation" (Ref. 20: 07708 
The specific tasks which are also assigned to COUMOPTEVEORAEN 
the CHO are summarized as: 


runction as an independent test agency for OT&E 
under the command of the CNO. 


0) 


b. Serve as principal advisor to the CNO for am 
Department of the Navy OTE macions 


c. Present results of OT&E to the JRMB at 1166557 
Ili (production decision) review and to other reviews 
as directed by tne CNO. 


d. Conduct operational tests on weapon systems 
includng ships and aircraft. Evaluate operational 
effectiveness, suitability; and capability; reporting 
the results to the CNO. 
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e. Assist development agencies in DICE, Med иии | 
support» as required», reporting the result sion 
assists and an assessment of Che sys£9m57— ee 

f. Review and evaluate the T&E planning for new weapons 
systems to address and resolve critical issues and 
report the findings to the CNO [Rep RR GS K 7 

The current authorized personnel Strength of OPTEW RR 
is approxinateT”777007 tvs Norfolk Headquarters has 268 
personnel, including 119 officers, 119 enlisted; "amm 
civilians [Ref. 21]. This represents only about one=fif Gime 
tne authorized positions in 0575777 The otner personnel 
are spread across the subordinate command (DEPCOHOPTEVE EO DB 
organic aircraft squadrons (VkX=-1,VX-4.,VX-5), and ma 
Sunnyvale detachment. Figure 8 shows the current 
Organizational arrangement ot OPIEVEORE 

It is interesting to note that when OPTEVFOR does OT&E 
on a ship or submarine it preforms the tests in a vem 
similiar manner to that of the Army or Air Force. OPTEVFOR 
usually sends a detachment ot irio rr ppp 77 to the 
ship/submarine to direct the testing, but utilizes MEN 
unit's precommissioning crew instead of there own personnes 
for conducting operations. In the case of an aireralt m 
related hardware, OPTEVFOR uses tne total in-nouse concept; 
using only their own personnel to test the item. 


Operational Test and Evaluation Force is presence 


managing OT&E for 92 major and non-major systems [Ref. 20]. 
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In this chapter the services' independent test agencies 
have been reviewed. Their organizational relationships 
methods of performing OT&E have been highiighted. A summary 
of these relationships and methods are provided in Appendix 


p 
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V. THE ADVANTAGES AND DISADVANTAGES OF THE 
SERVICES' INDEPENDENT TEST AGENCIES 


In response to DoD directives, each service has elected 
7 reuvc approach in tLhelr organizational structure for 
sProrming OTGE. Botna the Army and the Air Force chose to 
participate much more closely with the Developing Agency 
EU both OTEA-and ABOTEC are small organizations whien 
EE, direct» and control Ehe Operational testing, yet they 
Mee one actual testing to the user command, normally 
met lizings the unit programmed to receive the first opera- 
530nal system. 

The Navy» on the other hand, has elected to use the to- 
zü 71n-nous6 concept in the case of aircraft systems" test- 
m ahd evaluation. OPTEVFOR acts as a surrogate for the 
user by utilizing their own independent test facilities and 
57 Us to conduct tests on the systems. Additionally, OPTEV- 
FOR believes that the involvement with the DA will jeopard- 
ize their objectivity with the system [Ref. 22]. 

In trying to answer the question as to whether OPTEVFOR 
mesas be to restructured, it will be best to address the 
advantages and disadvantages of each service's independent 
Operational test agency. 

ME VOR ES 5020700007 NEARUFE TS TNEPr 1m5lstenee oH 


total independence in order to maintain their objectivity in 
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the T&E process. Currently, OPTEVFOR insists (2147070000 — 
to maintain their independence or objectivity; inte e EE 
between OPTEVFOR and the DA has to be kept to a minimum. 
Stated another way, the issue is independence versus 
cooperation, with OPTEVFOR placing more weight on them sm 
of independence. [Ref. 22] 

lt is this independence that is considered Tem 
strength because they do not have to concern themselves with 
the developer trying to twist their arm in an attempt Gomes 
a favorable decision. Additionally, by making themselves 
the tester and not using the operational commands, they are 
able to address the system witn more objectivity Since eames 
do not nave a vested interest in tne system. 

OPTEVFOR's insistence on independence is also their big- 
gest disadvantage. This reluctance has been the focal point 
in numerous studies, such as the the President's Blue". 
bon Panel Report (BRDP) in 1970. The BRDP believed that it 
is important to perform OT&E on an operationaliy conim 
production system. However, if the OT&E process only comz 
mences at OPEVAL, it misses most of the Opportunity stoma 
fluence the product during its development on behalf of the 
operational forces. [Ref.10: p. 88] The Defense Semenas 
Board on 17 February 1977, published a study statin:57 
"interaction among development test and evaluation and 


Operational test and evaluation, in addition to CNEJGIORE 
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255050 iL Lie Users. pays very important divwdends in 
terms of money, time and operational suitaoility"  [Ref. 1: 
25 71. Both pesce renos. alone: with OBNENS ina cate 
phat a working relationship between the DA and the indepen- 
BENE test agency 1s critical to the acquisition process. 

once OPREVFORSdJGES not actively participate in the 
system, this tends toO eredtem Ne mi Totha "Syndrome 
İ 0 1. Whienta system comes from TECHEVAL to OPEVAL, 
2:15 a possibility that it may not pass due to a change 
in OPTEVFOR test criteria. A weapon is developed to meet a 
2:66 operational requirement (OR). This requirement can be 
affected by funding reductions, schedule delays and contract 
disputes, in addition to changing threats new weapons and 
rew Lactics. As a result, the DA develops a system tnat 
fully meets the requirements of the original OR, but as a 
result of tne aforementioned ongoing changes, it falls 
“rt of the current OT&E criteria. Since OPTEVFOR does 
bobo Jaterface with the developing agency, it is extremely 
likely that the DA might not be aware of these changes. The 
200 result is the system is not ready for OPEVAL due to 
mest Criteria changes. IM MEUM O o in” “Una t ene 
1 06cha" takes place. 

Another study done in the summer of 1984 by the Naval 
75 --rch Advisory Committee (NRAC) further illustrates wnat 


mio results can be when there is little or no interface 
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between OPTEVEOR and те "DAR The committee reviewed a 
random sample of 142 of OPTEVFOR's OPEVACS CM 
conducted between 1975 and 1952. The results of tne study 
showed that 20% failed ORT NESE outright ande 
conditionaliy passed. These systems were assessed for their 
operational effectiveness and suitability. Of those Ghee 
failed. the majority of them failed the suitability WS 
of the test. [Ref. 251] The question which arises hens 
how can a system get this far in the process and then taa 
This is like having a student advance all tne way through 
tne education system and then give this student a Die mene 
test and find out tnat the individual is unable to readme 
write. 

in the author's opinion. the number of failure MM 
OPEVAL could be attributed to OPTEVFOR insaiscENCONAI 
maintaining their independence. With Inch ди 
communication, OPTEVFOR could alert the development agencies 
to the changes in the suitability requirements much car” 
in tne acquisition process. This in turn would reduce 
number of failures. By waiting until just before the system 
is ready to enter production to communicate them 
recourse if a discrepancy is discovered is top MM 
System.: 

On the other side of the spectrum : a Strone mpo m 


both the OTEA's and AFOTEC's arrangement is thein incre cma 
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5571070 Communicate vitnu tne DAs. Instead of representinz 
21131P.:n Lne casesci UPTEVFOR, BOLA OTEA and AFOTEC 
Breve in getting tne user involved at a very early staze 
INE uc İCE process. 0122 Uses a central group comprised of 
E wPp-ueonnel and"thewuser to test a system. The Air 
Force, on the other nand obtains the personnel needec to 
poene system from the user and support commands on either 
temporary Additlona. Duty (TAD) or Permanent Cnange of 
tation (PCS) basis. These personnel are then sent to AFOTEC 
momsupport tne program. 

OTEA nas recently implemented a new plan called 
Memtinuous Conception Evaluation (C2E), which has advanced 
the idea of continuous independent evaluation through tne 
E 5s . aegnmisition eycle. Differing from OTEA's previous 


Lol 


uEerniOJoFOEy, C2£8 begins at a much earlier point in the 
Es ic ion process and follows the system past the 
production pńase of the process. Tne data generated fron 
megels, simulations, contractor/government testing, and 
BRZ vv trials 1s now being collected and correlated for 
Mires uses This information will help to provide trends; 
BRO JESEIONS and inputs for existing and future projects. 
ELI oo) Thru €2E, OTEA is now in essence tracking a 
EDO" System from inception to retirement from inventory. 


mUre 9 shows the major projects that are currently under 


meee iRef. 23: p. 10]. 
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5377127?) 5770006 ” 00 01 in char it establishes a close 
I i OI thewbDA.Pne AMP Force's new empnasis on the 
operational requirement of a svstem nas resulted in a even 
closer working relationship between AFOTEC and the DAs. The 
ees WOW continually kept advised of any changes in the 
EX enm"s operat?*onali requirements by AFOTEC. The obvious 
Denefit is that there are no surprise OR changes when the 
57:57“n enters OPEVAL. Additionally, the Air Force, in an 
2 “Dt to shorten the acquisition process, is trying to use 
DT&E to nelp verify the operational performance of a system. 
meereduction in duplicative testing can be achieved since 
some of these tests will not have to be repeated under OT&E. 
2? lv being used on the B-1 bomber, this approach is 
considered a significant change and is still in the very 
Ep stages of evolution by the Air Force. [Ref. 24: p. 5] 

hrs elose working relationship of OTEA and AFOTEC with 
are DA and user commands can Shorten the acquisition 
process. By alerting the system to changes and» in the case 
Eu ne Air Force, reducing duplicative efforts in testing 
the total time required to field a system is reduced. 

The underlying reason behind the establishment of tne 
independent operational test agency (JTA) was to assure tne 
independence in the testing of a system. However, one of 


mee biggest disadvantages that can result from too close a 


working relationship between the ITA and the DA/user command 


ol 


is a Loss of Ob jeetauunn me it is very hard comma MN 
objeetivity when so much time has been spent with the people 
developing the system. Both OTEA and AFOTEC have to can: ии 
With pressures Inon notr only the DA out also tne user, 
Doth of  wnom are trying to get the system into production. 
The DA wants the system to enter production Dec aos ca ии 
maxes him look good. The user wants the hardware since it is 
perceived to be better than that whien he has presently. As 
a result, ne wants the system to do well in testing asm 
would expedite the fielding of the new system. If tne ITAs 
bends to their wishes, the result could be a system enter 
oroduction that might require extensive after-producwags 
modifications to make it function properly. Any sav NEN 
time and costs which were gained by the ITAS interfacing are 
now removed by the additional time and costs required to fix 
tne system's Snhorteconanas. 

In summation, this chapter has illustrated Mumm 
OPTEVFOR's strongest point. therr insi Se E E x 
independence, is also their weakest feature. In their effort 
to insure their objectivity, OPTEVFOR has missed Ru 
opportunity to influence the system's development On 564252 
of the operational forces, Additionally; their acim 
cooperation can add to the length of the acoui ir Rm 
process. By not keeping the DAs appraised of changes in 


the OR it is possible that a system might have CENE 


sri CO | development to correct the shortcominzs 
introduced by the new requirements. On the other nand, 
25 6 and ДРОТЕС "5 MS AS le ons 
27 1ONSMIP with che DA. Thais relationship can correct 
ии On S most prevalent shortcomings, novever, it in 
a ET? can introduce problems if carried too far, resulting 


EN Ne release Of a system that is not yet ready for 


peodguetion. 
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Vi. CONCLUSIONS/RECOMMENDATIONS 


Before ve put a weapon systema the hands of oum 
troops we should ensure that it performs 1ts combat mi 
Sion». not that it simply meets contract specifica mon 
: American service personnel have a right to ex= 
pect. that Tre government will provide them with 
necessary tools to do their job. [ret icon 
This statement made by Representive C. E. Bennett, 20 
excellent summation of what is expected from OT&E in the age 
quisition process. The purpose of the independent "WERSJE 
agencies is to determine whether tne weapon system fuifilis 
the desired function in an operational envYronmNemHb 
In the introduction, a statement from the Summer 1984 
NRAC report indicated that restructuring would enormous 
reduce the size and cost of OPTEVFOR. In addition Ra 
reporte stated "Phat the user's inputs could be Intro im 
into the process on an earlier and continuous basis [Ref.2]l. 
The first part of this statement is appealing, however the 
preceding analysis of operational test agencies indicates 
taar there is strong reason to believe that changes in 
operating policy, not the actual structure of T OPTi ni 
should be made. 
The present manning levels of OTEA and AFOTEC are in ac- 
tuality about the same or even greater than OPTEVFOR. ANIE 
analysis of the manning levels is baSed on counting thes mm 


sonnel that OTEA and AFOTEC utilize from other commands am 


support of their testing. By comparing these numbers agam ин 
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ZN PORTS Eotal eompiement of officers, enlisted and 
"RAGE Ens, becomes jevident that the Army and Air Force 
7 S “0107 €ffort is net less than the İlaVy"s. For ex- 
1:15 AFSOTEC's level ef 2500 supporting personnel is al- 
7 gGoudle that of OPTEVFOR total in-nouse complement of 
Il O personnel. 

Considering the costs of those services supplied by the 
other supporting commands, it appears tnat operating costs 
Om each ITA are conparable. ln essence, there is very 
fc le difference between the size and cost of each ITA if 
WIE Supporting commands for OTEA and AFOTEC are included in 
mae analysis. The actual cost of each ITA and its supporting 
command was beyond the scope of the research for this 
thesis; however, the potential value of this information 
I ..iurther study. 

The statement that the users inputs need to be provided 
earlier and on a continuous basis can be viewed as a result 
PROPTEVEOR's philosopny and not its existing structure. Un- 
SS recently, OPTEVFOR nas been resistant to the idea of 
Gu. acting more closely with the DAs, so that it could 
maintain objectivity. The importance placed by OPTEVFOR on 
Maintaining objectivity has changed somewhat witn tne new 
memmanding officer (COMOPTEVFOR). An examination of the 


findings of the Summer 1984 NRAC report reveals a change in 


35 


relationship between OPTEVFOR and the DAS [Ref 2217760 
nears that the new COMOPTEVFOR endorses increased liaison 
with tne DAs. 

In the past, when a system arrived for the OPEP 
that did not meet the test criteria it would minine ee 
fail. Now, if a problem is discovered, instead of falla 
the system outright, CO!IOPTEVFOR will call his counter 
at the DAs and inform him that the system is not doing wella 
This allows. the "DRE to remove the system from EES MES 
correct the deficiencies. This new approach nov on iO 
the problems in-house and away from the public, but it/asse 
fosters zreater cooperation. 

This cooperative relationship should not begin JUSTINEM 
the time and point that a system enters OPEVAL. It iS Woe 
enough to just advise the DAs that the system is not dGINIE 
well and needs some correction. The Navy needs to strongly 
encourage the early; continuous, and positive participat e 
of all key players as described in OPNAVINST 3960.10. The 
difference between the operational test requirements and 
that of development testing mandates closer coordination 
between the Navy's independent test agency and its develop- 
ing agencies. These organizations need early and clear 
definition of the test objectives, criteria and the SESNNIE 
ards. Once these conditions can be agreed upon, they must 
be kept current throughout the program to minimiz200 


necessity for repeat development and testing. Failure to do 
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77:75:02) result. in unnecessary delays and higher costs in the 
acquisition of fleet-ready weapon systems. 

Another area outside the scope of this researeh is the 
cost associated with having to send a system back through a 
EN of che acquisition process because it fails OPEVAL. 
tere are undoubteciy increased costs, however the magnitude 
e ais problem was not pursued. This subject has potential 
Matte in determining the real efficiency of the OT&E effort 
200 nerits further investisation. 

5 10558 Of independence in operational test is not being 
advocated nor is inference being made that objectivity is 
MOE important. [he real key to effective and efficient OT&E 
mommeaningful communication among the participants in the 
525 001sition process. If the test requirements are 
@eordinated and discussed by all parties, then conflict may 
Sm -dueed to minor proportions or eliminated altogether. 

2 EYFOEH SOWNO NOt ehanen its existing structural 
5“Fangement, since it been has demonstrated that this vas 
7 :nere the problem exists. Instead, OPTEVFOR needs to 
expound upon the new movement of increased communication 
DE the noc Me wend resule onn tnis increased 
sohimuńication will be a vastly improved independent test 
agency that may ameliorate the potential of an inadequate 


7 “mn reaching the fleet. The Navy can neither afford 
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the development of systems that do not meet the fleets 
needs; nor can they afford the publicity that” results 79 


produced system that fails in the fleet. 
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RPPELDIX A: 


ACAT 


AROTEC 


AFR 
AFSC 
AMC 
AR 
ASARC 


ASD(ASL) 


ASNCR&D) 


BLS 


BADPR 


SOHOPTEVFOR 


CSA 


DCP 


GLOSSARY OF ACRONYMS AND ABBREVIATIONS 


üounisitron Category 


ino Ponce Operational test and Evaluation 
Center 


HIT PORESNASCUIaTION 

Air Force Systems Command 

Army Material Command 

Army Regulation 

Army System Acquisition Review Council 


Assistant Secretary of Defense ( Acquisition 
uddu res) 


Assistant Secretary of the Navy for Research 
and Development 


Board of Inspection and Survey 

Blue Ribbon Defense Panel Report 

Combat Development Experimentation Command 
Chief of Navy Operations 


Commander», Operational Test and Evaluation 
Force 


Chief of Staff Army 

Chief of Staff Air Force 
Coordinated Test Program 
Demonstration and Validation 
Developing Agency 


Decision Concept Paper (NAVY) or Development 
Concept Paper (ARMY) 
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DCEZ ICE 


DCSRDA 


Lip 


DDR&E 


DEPCOHOPTEVFOR 


DoD 

DoDD 
DOT&E 
DSARC 
BT 

DE ZE 
ADT 
BOLu 
BLİ 


GAO 


IOT&E 
JMSNS 
J RMB 
COL 
MAC 


МАЈСОМ 


Decision Coordinating Paper/ Integrated 
Coordina tama Paper 


Deputy Chief of Staff Research Development and 
KEGUNSWE REM 


Deputy Director of Defense Research and 
Engineering, Test and Evaluation 


Director of Defense Research and Engineering 


Deputy Commander Operational Test and Evaluation 
Force 


Department of Defense 

Department Of Defense Directive 

Director, Operational Test and Evaluation 
Defense Systems Acquisition Review Council 
Development Testing 

Development Test and Evaluation 

Force Development Testing and Experimentation 
Follow-on Test and Evaluation 

Five Year Test Plan 

Government Accounting Office 

Integrated Logistics Support 

Initial Operational Capability 

Initial Operational Test and Evaluation 
Justification of Major Systems New scans 
Joint Requirement and Management Board 
Letter of Instructions 

Military Airiift>conmenie 


Major Command 
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AS PER 


NAVATR 
OPEVAL 
OPNAV 
OPNAVINST 
APTEVFOR 
OR 

OSD 

OT 


eh 


PATGE 
ELS 
PDA 
PDM 
Du 
FNP 
POM 
FPBS 
RAM 
RDT&E 
RDT&E 
ROC 


SAC 


бе Aku election Systems Test Evaluation 
Review 


Naval Air Systems Conmand 

Operational Evaluation 

Office of the Chief of Navy Operations 
dice no onie! OfeNavVal Operations instruction 
Operational Test and Evaluation Force 
Operational Requirement 

Office of the Secretary of Defense 
Operational Testing 

Operational Test and Evaluation 
Operational Test and Evaluation Agency 
Production Acceptance Test and Evaluation 
Permanent Cnange of Station 
EzozraulDecision authority 

Program Decision Memorandum 

Program Manager 

Program Management Plan 

Program Objective Memorandum 

Program, Planning and Budget System 
Reliability, Availability, Maintainability 
Research, Developnent, Test and Evaluation 
Research, Development, Test, and Evaluation 
Required Operational Capability 


Strategic Air Command 
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SECNAVINST 
SOR 

SPO 

TAC 

TAD 

Poe 

TD 

TDP 
TECHEVAL 
ECOS 
PENE 
TRADOC 
USACDC 
USD (A) 


VX 


Systems Acquisition Management 

System Concept Paper 

Secretary of Defense Decision Memorandum 
Secretary of Defense 

Secretary of the Navy Instruction 
Specific Operational Requirement 

System Program Of: es 

Tactical Air Commana 

Temporary Additional Duty 

Test and Evaluation 

Test Directive 

Technical Development Plan 

Technical evalunatran 

Test and Evaluation Command 

Test and Evaluation Haster Plan 

Training and Doctrine Command 

U.S. Army Combat Developments Command 
Under Secretary of Defense for Acquisition 


Air Test and Evaluation 5quadron 
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APPENDIX 


US AIR FORCE 


1. Designation of 
major field agency 
responsible for 
OT&E: 


Air Force Opera- 
nal Test and 
Evaluation Center 
MIO TEC) HO AF TEST 
AND EVALUATION CEN- 
FER 

irlanda Air 

Force Base 

New Mexico 87115 


Autovon: 244-0545 
commercial: (505) 
264-0545 


2. Manning Level: 
(authorized) 


507 
110 


Meer i tary 
Civilian 


3. Reporting Chain: 


MepDorts direct to 
onner Staff of the 
iue Force. 


Ub. Principal 
directives/regula- 
tions concerning 
OT&E: 


AER 30-184 "Test 
Anar Evaluation" 
D AFR 23-36 "Air 


Force Operational 
Mest and Evaluation 


Mater (AFOTEC)" 


BE 


US ARMY 


U 5 Army Operational 
Test and Evaluation 
Agency (OTEA) CG, HQ 
USS shinee er bead TONAL 
DET ance NONI UA TION 
AGENCY 
5037505 VA: 
Autovon: 289-2228 
Commercial: (703) 
756-1254 


1530 
126 


Military 
Cav an 


Reno Ls dire: Lt. Fo 
Chief*otraFr=6of. tlie 
Army. 


a. AR 70-10 "Test 
am dwm evaluat on 
During Development 
and ACGUTSTŁION of 
Material" 


25577 2/17 AIR FORCE COMPARISON 


US NAVY 


Operational Test and 
Be yarn teow Of Ce 
COP PEE OR) CZARA = 
MANDER, OPERATIONAL 
BED ERI DATE 
FORCE 
Погто1 VA. 2351 
Ructowone 501-297 
Commercial: (804) 
444-5337 


238 
30 


Military 
Civilian 


REDORES direct. to 
Chief of Navy Opera- 
PARODNSZ 


a. SECNAVINST 5000.1 
"System Acquisition" 
D. OPNAVINST 3960.10 
MESS anA ENa l -= 
patron" 


5. Field activities 
directly under OT&E 
Test Agency: 


all facilities c up: 
port and personnel 
support provıdedley 
designed MAJCOMS. 


6. Comparison of 
characteristics of 
OT&E: 


a. Development of 
OPEL иси, 
techniques, proce- 
dures and concepts. 

07 EVO Dita ono 
reliability, main- 
Lainability, and 
operational effec= 
tiveness and 
SUPL ADELNCYZ 

c. bkestine under 
realistic opera- 
tional conditions.: 

dz cconpiish 1015 
prior to first mor 
production decision, 

e.FOT&E subsequent 
boo: . “22666 0 DL 
production itens: 

[o LOTGE will fnot 
obviate the need for 
FOT&E. 


b. AR 10-4 "US my 
Operational Test and 
Evaluation Agency" 


None. OTEL emma 
tributes deputy test 
director ands toms 
key personnel to 
test at the command 
cońnductiINESRESSU. 


a. Representative 
user troops 

o. Realistic opera- 
tional environment 

c. Providesydises? ite 
estimate military 
utility, operational 
effectiveness, and 
op era t) om 
suitapilliy?in- nz 
ing “compatir oi Eey 
interoperada weve 
re 1982077301 e: 
availaoility main- 
tainability ipo 
tegrated logis tio 
support and "pain 
requirements. 

d. IOT&E examines 
the hardware or com- 
ponents to provide 
an indication 
utility and ört ra 
the user. 
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c= OPNAVINST 
5440.47 "Missl0n ama 
Function of OpSTSM 
tional Tes P RE 
Evaluation Force" 


a. Three Air, Test 
and Evaluation 524222 
(VX-1, 1, ang p 
b. Deputy COMORES 
FOR San Diego 
c. New London Test 


and Evaluate 
Detachment 
a. O TG par- 


ticipated ın 0702522 
formed by opera- 
tional personnel, 

b. Focus e m 
operational effec- 
tlveness and 
suitability inem nm 
ing reiiab ilm 
compatibility, in- 
teroperaDil 777292 
maintainability and 
supportabili:77 

c. Development of 
optimun, operational 
tactic s 

dz lIOT&E a GR 
complished pricy ime 
JRMB ILl or e om 
parable CNO major 
production dec15122 
pommes 








Ли 77 ociüses on 
the operational and 
employment aspects 
Sieve “system. 

AFR 80-14 


7. Authority for as- 
signment of OT&E 
projects: 


s hief Staff of 
mae Air Force (CSAF) 
meer OLlEC may 
mecommend to CSAF 
hose OTEE projects 
“bich it or MAJCOMS 
550010 conduct. 


8. Master test docu- 
ment for specific 
(major or selected 


non-major) IOT&E 
program: 
section 5 (Test and 


2:101:stion) of 
Program Management 
pian (PMP). 


9. Views on combined 
DT&E/IOT&E: 


a.Operating and 
Supporting commands 
Pepe bit cipation is 
Meda atory during 
development testing. 


775 ia on of 
equipment prior to 
un production 
deca sion: 
"HOME 
prior o 
AR 70-10 


Final check 
acceptance. 


Be Pu C ye hler of 
Staff Research 
Development and Ac- 
quisition determines 
the major and non- 
Wea jor Systems: 


Coõoordinated Test 
Programi (CTE) 


a. The OT&E process 
will be independent 
and should normally 
be separate from the 
DT&E process. 
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e. Assessment of 
operational effec- 
¡A AS 2 and 
SUT Babi ey. 

f. FOT&E conducted 
in operational en- 
vironment by opera- 
tional personnel 
(S AO duc tion 
Systems. 

g. Refine tactical 
employment doctrine 
and requirements for 
personnel and train- 
TAS 

OP TAY Gis 3900-10 


a. Chief of Naval 
Operations (CNO) 

De. Фи 149 ma 
authorize COMOPTEV- 
FOR to further as- 
5577080 Opec FS LO 
otner commands when 
advantages are ap- 
parent. 


Test and Evaluation 
Mas Len mba TPE TP) 


a. OPTEVFOR par- 
b PE es Um n the 
HİD 1567700 İCE) 
EE EE 
conducted by the 


b. Operational in: 
puts will be util- 
ized in (test) plan- 
ning aoe Uneme 
developed by the im- 
plementing Command. 

c. To maximum poss 
Sible- degree, 121: 
Will be accomplished 
completed by operat- 
ing personnel of the 
Operational’ Supe 
porting Commands 
using reali Tee 
operationai environ- 
ment. 

d. Where adequate 
test data can be 
Sec ted) Tron came 
bined DIXE/IOTGE> 


combined tests may 
be employed. Added 
considerations Tor 


combined testing are 
costs and time (AFR 
30-14). 

e. Separate evalua- 
tions required from 
Operating, suv 0ertineg 
Commands in Produc- 
tion VeCcisSions. 


10. Document chain 
in acquisition 
system: 


a, PROC (tormally 
states a Required 
Op: e r a a © mas 
Capability) 

b” orionu Direc: 
tives (early PMD) 

eC. program Advocacy 
Documents 

dis. DUCE OTE. inputs 
prov dedi oy ~ArOT Be) 

PAD Sd nl taa 
Test Directive (ORCE 


Hovever, it may be 
combined u nen 
separation causes 
delay involving un- 
acceptable Fisk GM 
unacceptable ac- 
quisition costs =E n 
70-10). 

D. Te stum v ss 
usually phased as DT 
I 11, anad III and 
OT I, Tl ip o papi m 
Usually кии 
in combinatkson "UCD 
DT I. Armymattenpts 
to keep; Im ma q 
III and DEM na 
IIl separated. 

c. Each pias > 071 
testing results in a 
separate evaluation 
by OTEL” rue O 
provide evaluation 
Ee, COto om c is im 
body (JRMB) at deci- 
sion milestones: 

d. SDT NM TT ES 
and DT III are con- 
duc Loe a el 
development items/ 
systens as a mini- 
mum. 


a. ROC 

b. Charter for spe- 
cial task force 

c. OFEA evaluation 
plan 

d. Program Advocacy 
Document 

e. DCP (OT&E inputs 
provided by OTEA) 

f. Final RSDOMREJGSN 
Special Task Force 
(contains plan ior 
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Developing Agency 
with OPTEVFOL 3 
puts. OPTE?77 2 
provides independent 
operational assesss* 
ment to the CHEM 

b. ORIENT 
responsible m 
planning», conduct- 
ing, repo r os 
OPEVAL, conduct 
after TECHEVALSERN 
prior to RH po 
Decision milestone. 


a, SÜR (formalı 
states a Specif?2 
Operation ai 
Requirement) 

DE Techn i clam 
Development PIG 
(TPD)- (program ad- 
vocacy document in- 
cluding OT&E inp 
by OPTEVFORƏ 

c. OPTEVFOR evalua- 
LION ии 








ити provided Dy 
AFOTEC) 

f. Test objectives 
23255 to the PMD 
SRO USAF Test 
Directive (Drafted 

5 y FO TE C ) 
5. EBRO TE € 
Commander's Estimate 
U :.EC Test Plan 
j. Program .lanage- 
ment Plan (contains 
Mick, OT&E tests 
plans) 

ie interim Test 
reports (interim 
evaluation) 
i Fina ti 
Report 

m . 5500 AD EC 
Commander's Summary 
(Final independent 
ops evaluation) 


Test 


11. Commands con- 
ducting OT: 


EIA EDO TEC=" (all 
major and selected 
nonmajor systems) 

b. Designed AF 
MAJCOMS- (nonmajor 
Systems) 


OT&E 
OTEA) 
cnn [P Or Ti DOC 

Outline Tesc Plans 

Jo L Plans 

Development Plan 
Mero laica S. C GO O: r = 
dinated Test Program 
AD. anda TOT 
pre sitas of “ne 
system) 

go L/DI-T Test 
Reports (independent 
operational evalua- 
Con 0727 REA) 

5757007501 DT LE Test 
Reports (independent 
operational evalua- 
Cron by O PEA) 

II OA ADT TEL 
Test Rop orts 
(independent opera- 
tional evaluation by 
OTEA) 


provided by 


a. OTEA- (all major 
and selected non- 
major systems) 

b. Combat Develop- 
ment Experimentation 
command CDEC) = 
(nonmajor systems) 

c. Modern Army 
Selected System 
Evaluation and 
Reviev (MASSTER)- 
(nonmajor system) 

d. Other Major Com- 
mamas ” (nonnayor 
systems) 


2550106 inputs 
pO: occ d O V 
OPTEVFOR) 

e. Test and Evalua- 
uronuester Plan 
Cre TEDER (contains 
DT&E and OT&E plans 
[Jos исә ис 1 İİ C 
program 

1500060166 7 Paster 
basan CuB P ) = 
contas "cane TENP) 


Вә РС ДЕ En 
(the developing 
azeney plan wor 
DT&E) 


IE SENA e pom 
Jos p TEVEOR S indee 
pendent assessment 
о тест уд, 
jee ORs VAT 
ƏL 

k. OPEVAL Report- 
(independent opera- 
tional evaluation by 
OPTEVFOR) 


601255 


CO Or OR 


12. Commands per- 
forming evaluation 
of OT&E directed by 
Service HQ: 


The command conduct- 
ing OT perrorms "enie 
evaluation subject 
to review, approval: 
and further evalua- 
tion oy AFOTEC3 


ОГЕД регт әт 
evaluation. 
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the evaluation. 
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